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ABSTRACT 

The  existing  spatial  smoothing  (SS)  technique,  although  it  is  effective  in  decorrelating  coherent 
signals,  can  only  be  applied  to  uniformly  spaced  linear  arrays  which  are  very  sensitive  to  the 
directions-of-arrival  (DOA’s)  and  can  be  used  to  estimate  azimuth  angles  only.  To  significantly 
improve  the  robustness  of  DO  A  estimation  and  of  beamforming  and  to  estimate  both  azimuth  and 
elevation  angles,  we  developed  techniques  for  applying  SS  to  arrays  of  arbitrary  geometry.  We 
found  that  an  array  must  have  an  orientational  invariance  structure  with  an  ambiguity  free  center 
array  for  applying  SS.  We  also  study  the  cause  of  ambiguities  in  a  multiple  signal  environment  and 
find  the  necessary  and  sufficient  conditions  for  an  array  manifold  to  be  ambiguity  free.  If  an  array 
is  also  central  symmetric,  the  forward/backward  spatial  smoothing  can  be  used  to  improve  the 
resolution.  Finally,  we  expand  the  application  of  our  technique  not  only  to  MUSIC  and  adaptive 
beamforming  algorithms  but  also  to  ESPRIT  algorithm.  All  the  predicted  results  are  verified  by 
simulations. 


This  work  was  supported  in  part  by  the  NSF  grant  MIP-93-09-506,  the  ONR  grant 
N00014-93-1-0566,  and  MIPS/Watkins  Johnson. 


I  Introduction 


Sensor  array  processing  has  been  a  key  technology  in  radar,  sonar,  communications  and  biomed¬ 
ical  signal  processing.  Recently,  as  the  cellular  communication  technology  advances,  sensor  array 
processing  emerges  as  a  potential  technology  to  improve  the  spectral  efficiency  [1],  [2],  [3]. 

Much  of  the  work  in  array  processing  has  focused  on  methods  for  high- resolution  DOA  estima¬ 
tion  and  optimum  adaptive  beamforming.  These  methods  include  the  well  known  MUSIC  [4]  algo¬ 
rithm  and  ESPRIT  [5]  algorithm  for  DOA  estimation  and  MVDR  and  LCMV  algorithms  [6],  [7],  [8] 
for  beamforming.  However,  an  important  drawback  of  these  techniques  is  the  severe  degradation  of 
the  estimation  accuracy  in  DOA  estimation  [9]  or  signal  cancelation  [10]  in  adaptive  beamforming, 
in  the  presense  of  highly  correlated  or  coherent  signals. 

To  counter  the  deleterious  effects  due  to  some  coherent  signals,  a  pre-processing  scheme  referred 
to  as  spatial  smoothing  (SS)  proposed  by  Evans  et  al.  [11]  and  further  developed  by  Shan  et 
al.  [9],  [12]  has  been  shown  to  be  effective  in  decorrelating  coherent  signals.  However,  such  a 
scheme  can  only  be  applied  to  uniformly  spaced  linear  arrays.  Linear  arrays  are  known  to  be 
limited  to  estimating  azimuth  angles  within  180°,  and  practically  effective  only  for  signals  from 
broadside  direction.  The  degree  of  SS  using  a  uniformly  spaced  linear  array  is  also  sensitive  to 
DOA’s  [13].  As  a  result,  a  linear  array  is  not  very  effective  in  radar,  sonar,  or  especially  in  cellular 
communications  where  users  can  never  predict  the  incoming  directions  of  the  moving  targets. 

In  the  past  decade,  researches  have  been  carried  out  in  developing  algorithms  for  coherent  in¬ 
terference  using  arrays  of  arbitrary  geometry.  In  the  area  of  DOA  estimation,  multidimensional 
subspace  fitting  algorithms  such  as  deterministic  maximum  likelihood  (DML)  [14],  multidimen¬ 
sional  (MD)-MUSIC  [15],  and  recently  proposed  weighted  subspace  fitting  (WSF)  [16],  [17],  are 
effective  in  both  coherent  and  noncoherent  environment  and  can  be  applied  to  arrays  of  arbitrary 
geometry.  However,  all  these  algorithms  involve  some  searching  procedures  used  to  solve  nonlinear 
equations.  They  are  computationally  intensive  and  are  not  practical  in  real-time  applications.  In 
the  area  of  narrow-band  adaptive  beamforming,  The  coherent  interference  suppression  using  null 
constraint  with  an  array  of  arbitrary  geometry  was  addressed  in  [18].  This  approach  still  requires 
pre-estimation  of  arrival  angles  of  coherent  interferences.  Recently,  the  SPT-LCMV  beamforming 
algorithm  applicable  to  arrays  of  arbitrary  geometry  was  considered  in  [19].  This  algorithm  requires 
increased  computational  complexity  compared  to  LCMV. 
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In  this  work,  we  develop  a  general  SS  technique  for  arrays  of  arbitrary  geometry  to  make  MU¬ 
SIC,  ESPRIT  algorithms  and  optimum  adaptive  beamforming  algorithms  operative  in  a  coherent 
interference  environment  and  meanwhile  achieve  robustness  in  performance.  Compared  with  the 
aforementioned  methods  for  arrays  of  arbitrary  geometry,  this  SS  technique  can  be  easily  imple¬ 
mented.  It  does  not  increase  the  computational  complexity  of  either  MUSIC,  ESPRIT,  or  adaptive 
beamforming.  It  allows  us  to  work  on  a  data  domain  [20],  and  thus  enables  us  to  incorporate  the 
recently  developed  URV  [21]  [22]  algorithm  to  DOA  estimation  and  updating  and  enables  us  to 
implement  MVDR  beamforming  algorithm  using  systolic  arrays.  Therefore  it  has  great  potential 
in  mobile  radio  communication  where  coherent  and  cochannel  multipath  interference  is  a  major 
problem.  Also,  it  can  be  used  in  conjunction  with  MUSIC  or  ESPRIT  algorithm  to  provide  an 
initialization  for  the  WSF  method  to  get  a  more  accurate  DOA  estimation  [17]. 

Specifically,  we  discovered  and  proved  the  conditions  on  an  array  of  arbitrary  geometry  for 
applying  SS.  They  are:  (1)  such  an  array  must  have  an  orientational  invariance  structure;  (2)  its 
center  array  has  an  ambiguity  free  array  manifold;  and  (3)  the  number  of  subarrays  is  larger  than  or 
equal  to  the  largest  number  of  mutually  coherent  signals.  By  working  on  a  smoothed  data  matrix 
obtained  from  SS,  we  can  use  MUSIC  and  optimum  adaptive  beamformers  effectively  in  a  coherent 
interference  environment.  To  further  increase  efficiency  and  estimation  resolution,  we  found  that 
the  forward/backward  spatial  smoothing  [23]  (FBSS),  when  applied  to  a  nonlinear  array  of  central 
symmetry,  can  reduce  the  number  of  sensors  required  and  improve  the  estimation  resolution  for 
closely  spaced  incoming  signals.  Finally  we  expand  the  application  of  our  results  to  ESPRIT. 

In  all  the  papers  cited  above  that  dealt  with  DOA  estimation  with  arrays  of  arbitrary  geometry, 
ambiguity  free  array  manifolds  were  assumed.  In  [4]  Schmidt  discovered  and  defined  the  rank-n 
ambiguity  in  an  array  manifold.  However,  no  more  studies  on  such  ambiguity  had  been  reported  in 
the  open  literature  [24].  Then  in  [24],  Lo  and  Marple  proved  the  conditions  for  a  rank-2  ambiguity. 
In  this  paper,  we  report  a  more  thorough  study  on  this  issue.  We  proved  the  necessary  and 
sufficient  conditions  for  a  three-sensor  array  manifold  to  be  ambiguity  free.  We  then  identified 
several  situations,  for  higher  order  sensor  array  manifolds,  in  which  ambiguity  may  arise.  Thus  we 
get  corresponding  necessary  conditions  to  design  ambiguity  free  center  arrays  and  subarrays. 

This  paper  is  divided  into  six  sections.  In  section  II,  we  prove  the  necessary  and  sufficient 
conditions  on  an  array  of  arbitrary  geometry  for  applying  SS,  and  consider  the  FBSS  technique 
for  applications  in  nonlinear  arrays.  In  section  III,  we  study  the  cause  of  ambiguities  in  a  multiple 
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signal  environment.  In  section  IV,  we  present  some  simulation  results  and  show  that  dense  square 
arrays  have  a  preferred  geometry  for  applying  SS.  In  section  V,  we  expand  our  results  to  ESPRIT. 
Section  VI  concludes  our  work. 

II  SS  for  Array  of  Arbitrary  Geometry 

We  first  assume  that  all  the  sensors  in  an  array  discussed  in  this  paper  are  omnidirectional  and 
identical.  Consider  an  array  of  p  sensors.  Let  d  narrow-band  signals  with  additive  white  Gaussian 
noise  impinge  on  the  array  at  incident  angles  #i,' ••,#<£•  The  array  output  covariance  matrix  has 
the  form  [20] 

R  =  E(r{t)rH{t))  =  ARSAH  +  a2I.  (1) 

where  r (t)  is  the  received  signal  by  the  array  at  time  t,  A  is  a  p  X  d  steering  matrix  and  a2  is  the 
variance  of  the  white  Gaussian  noise.  When  there  are  coherent  interferences,  the  signal  covariance 
matrix  Rs  is  no  longer  full  rank.  Therefore,  all  the  high  resolution  DOA  estimation  methods  based 
on  eigendecomposition  and  all  the  adaptive  beamforming  algorithms  which  assume  that  interfering 
signals  are  not  fully  correlated  with  the  desired  signals  fail  to  operate  effectively. 

In  the  case  of  a  uniformly  spaced  linear  array,  with  a  sensor  spacing  A,  the  SS  [9]  [12]  algorithm 
can  be  applied  to  achieve  the  nonsingularity  of  the  modified  covariance  matrix  of  the  signals. 
This  technique  begins  by  dividing  a  uniformly  spaced  linear  array  of  L  sensors  into  K  overlapping 
subarrays  of  size  p,  with  sensors  {1,  •  •  -,p}  forming  the  first  subarray,  and  sensors  {2,  •  •  -,p  -f  1} 
forming  the  second  subarray,  etc.  It  was  shown  that  [9] 

Ak  =  A\E^k~l\  (2) 

where  Ak,  k  =  1,  •  •  • ,  K ,  is  a  p  x  d  steering  matrix  consisting  of  steering  vectors  associated  with  the 
kth  subarray,  and  E W  denotes  the  kth  power  of  a  d  x  d  diagonal  matrix  E. 

The  spatially  smoothed  covariance  matrix  is  defined  as  the  average  of  the  subarray  covariances: 

1  K 

R=-YJRk  =  A1RsA»  +  a2I,  (3) 
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where  Rk  is  the  covariance  matrix  associated  with  the  kth.  subarray,  Rs  is  the  modified  covariance 
matrix  of  the  signals,  and  has  been  proved  [9]  to  be  full  rank  when  K  >  d.  The  signals  are  thus 
progressively  decorrelated  [13].  However,  linear  arrays  have  limitations  in  the  domain  of  estimable 
DOA’s.  It  has  been  shown  in  ref.  [25]  that  Rs  can  be  decomposed  as  follows: 

R.  =  jcc»  (4) 

where  C  —  PAT  with  P  =  diag(pi,p2,  •  ■  • ,Pd),  and 

1  1  •••  1 

sin(0i )  e-i27rfsin(e2)  ...  e-j27r^-  sin(0d) 

A  —  e-i47rfsin(^l)  e-i47r^sin(e2)  ...  sin(^d) 

e-j2Kn& sin(fli)  e~j2K-ir^  sm(62)  ...  e~j2Kir& sin(8d) 

When  incoming  signals  are  closely  spaced,  the  columns  of  both  A  and  A\  become  almost  linearly 
dependent  [25].  The  dependency  increases  drastically  when  some  of  sin(Oi),  i  =  1,  •  •  • ,  d  approach  1 
for  DOA’s  near  90°.  As  a  result,  the  performance  of  a  linear  array  deteriorates  quickly  when  some 
of  DOA’s  approach  90°.  The  highly  directional  sensitivity  of  the  linear  array  causes  the  lack  of 
performance  robustness  to  the  DOA’s  and  limits  the  domain  of  estimable  angles  to  azimuth  angles 
from  broadside  direction  of  the  array.  The  lack  of  performance  robustness  of  a  linear  array  is  even 
more  severe  when  SS  technique  is  applied,  because  in  the  smoothed  covariance  matrix,  not  only 
the  steering  matrix  A\,  but  also  A  is  ill-conditioned  in  the  situation  described  above.  A  general 
SS  technique  that  is  robust  and  can  be  applied  to  directionally  independent  arrays  is  thus  more 
desirable. 

A  Orientational  Invariance  Structure 

It  is  apparent  that  the  mapping  relation  between  Ak  and  A\  is  the  key  to  successful  application  of 
the  SS  technique.  In  general,  we  can  divide  an  arbitrary  array  into  K  subarrays  which  may  overlap. 
There  is  not  always  a  steering  matrix  A  to  map  all  the  steering  matrix  Ak,  for  k  —  1,  •  •  •,  K  to 
A.  In  this  section,  we  will  develop  necessary  and  sufficient  conditions  on  array  geometries  for 
implementing  the  general  SS.  First,  we  give  the  following  lemmas. 
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Lemma  1  For  steering  matrices  A  and  B,  given  by  A  =  [a(0i),  •••,  a {Od)\  and  B  =  [b(0j);  •  •  •, 
b(^)],  there  exists  a  mapping  relation  B  =  AC  if  and  only  if  C  is  a  diagonal  matrix. 

Lemma  2  For  K  steering  matrices  A\ ,  A2,  •  •  • ,  Ak,  each  At  can  be  mapped  to  a  steering  matrix  B 
if  and  only  if  there  exists  a  mapping  relation  Aj  =  AtCtJ  for  any  i  and  j . 

The  proofs  of  both  lemmas  are  given  in  the  Appendix. 

Consider  an  array  that  is  divided  into  K  subarrays.  Suppose  Ai  and  Aj  are  the  steering 
matrices  associated  with  the  ith  and  the  jth  subarrays,  and  there  are  d  signals  with  incoming 
angles  6\,  •  •  • ,  6d.  A{  can  be  written  as 

Ai  =  [a,'(#i),a;(#2)>  •  •  • ,  a8(6»d)]  (6) 

where  (Ok)  =  \e~^a^k\e~^'2^k\  •  •  •  ,e~^lP^k^],k  =  1,  •••,</,  is  the  steering  vector  associated 
with  the  ith  subarray,  and  <f>u(9k),  /  €  {1,  -  -  -,p},  is  the  phase  delay  of  the  &th  signal  at  the  Zth 
sensor  of  the  ith  subarray  from  the  first  sensor  of  the  first  subarray.  We  refer  to  the  sensor  of  an 
array  associated  with  the  Ith  row  of  a  steering  matrix  of  the  array  as  the  Ith  sensor  of  the  array. 

Let  A tji,  1  <  l  <  p,  represent  the  distance  between  the  ith  sensor  in  the  ith  subarray  and  the 
/th  sensor  in  the  jth  subarray.  Let  /3iji  represent  the  angle  of  the  line  on  which  these  two  sensors 
are  located.  If  the  ith  and  the  jth  subarrays  are  identical  and  have  the  same  orientation,  i.e.  all 
A iji  for  l  =  1  ,•••,£>  are  equal  and  all  f3iji,l  =  !,•••,£>  are  equal,  then  the  phase  delay  of  a  signal 
with  an  incoming  angle  6k  from  each  sensor  in  the  ith  subarray  to  the  corresponding  sensor  in  the 
jth  subarray  is  the  same  according  to  the  far  field  assumption.  We  denote  this  phase  delay  by 
$ij(6k).  For  any  l  G  {1,  •  •  •  ,p},  we  have 

$ij(6k)  =  4>ji(6k )  -  <j>u(0k)  =  sin(Ai/  -  6k  +  |),  (7) 

then  Aj  —  A{C{j ,  where  C\j  is  a  diagonal  matrix  with  the  mth  diagonal  element  (e?n^ .  The 

identical  and  orientational  invariance  properties  between  two  subarrays  guarantee  a  mapping  rela¬ 
tion  between  their  steering  matrices. 
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On  the  other  hand,  if  Aj  =  AiC,  by  Lemma  1,  C  should  be  a  diagonal  matrix  and  can  be 
represented  by  C  =  diag{cn(di),c22(62),  -  •  ■  ,cdd(6d)}.  It  requires  that 

e-^k)Ckk(ek)  =  e-j(t>^k)  for  /  =  (8) 

which  can  be  simplified  to 

4>jl(8k)  ~  4>u{0k)  =  +  27rn,  for  l  =  1,  •  •  -,p  (9) 

where  n  can  be  any  integer.  The  relation  in  (9)  holds  for  all  6k  in  [0, 360)  only  if  A,-ji  =  Aij2  = 

•  •  •  =  A ijp  and  /3zyi  =  (3ij2  =  •  •  •  =  (3ijp ,  i.e.  the  ith  and  the  jth  subarrays  must  be  identical  and 
have  the  same  orientation.  Thus,  we  have  Lemma  3: 

Lemma  3  Suppose  Ai  and  Aj  are  steering  matrices  associated  with  the  ith  and  the  jth  subarrays. 
The  sensors  in  each  subarray  are  numbered  in  the  same  sequence.  There  exists  a  mapping  relation 
Aj  =  AiCij  if  and  only  if  the  ith  and  the  jth  subarrays  are  identical  and  have  the  same  orientation. 

From  Lemmas  2  and  3,  we  have: 

Theorem  1  Suppose  an  array  can  be  divided  into  K  subarrays,  each  having  a  pxd  steering  matrix 
Ai,  i  —  1, 2,  •  •  • ,  K.  All  A\,  A2,  •  •  • ,  Ak  can  be  mapped  to  a  p  X  d  steering  matrix  B  by  Ai  =  BDi 
if  and  only  if  all  these  subarrays  are  identical  and  have  the  same  orientation. 

We  call  the  array  structure  held  by  an  array  satisfying  conditions  in  Theorem  1  the  orientational 
invariance  structure.  A  more  rigorous  definition  is  given  as  follows: 

Definition  1  (Orientational  Invariance  Structure)  An  array  has  an  orientational  invariance 
structure  if  it  can  be  divided  into  subarrays  that  are  identical  and  have  the  same  orientation. 

For  an  array  with  orientational  invariance  structure,  we  can  consider  each  subarray  as  one 
element  located  at  its  first  sensor.  Then  all  these  elements  form  a  center  array.  A  more  rigorous 
definition  for  center  array  is  given  as  follows: 

Definition  2  (Center  Array)  If  an  array  with  orientational  invariance  structure  is  divided  into 
subarrays  (which  can  have  overlap),  then  the  collection  of  all  the  first  sensors  of  these  subarrays 
form  a  center  array. 
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B  Necessary  and  Sufficient  Conditions 

Suppose  an  array  has  an  orientational  invariance  structure.  Moreover,  its  center  array  has  an 
ambiguity  free  structure  and  the  number  of  subarrays  is  larger  than  or  equal  to  the  largest  number 
of  mutually  coherent  signals.  The  p  X  d  steering  matrices  Ai,  A2,  •  •  • ,  are  associated  with 
the  subarrays  1,2,  •  •  -  ,K,  respectively,  and  dk  is  the  distance  between  the  first  sensor  in  the  first 
subarray  and  the  first  sensor  in  the  kth  subarray.  The  angle  f3k  represents  the  direction  of  the  line 
on  which  the  first  sensor  in  the  first,  subarray  and  the  first  sensor  in  the  &th  subarray  are  located 
(see  Fig.l).  We  have 

Ak  =  A1Dk ,  k  =  2  (10) 


where 


Dk  = 


sin(/3fe-0i  +  f ) 


sin(0k-e2  + f ) 


(ii) 


sm(pk-dd+z) 


The  covariance  matrix  of  the  fcth  subarray  is  thus  given  by 


Rk  =  A1DkRaD%  A?  +  a2I,  (12) 

where  Rs  is  the  covariance  matrix  of  the  source.  The  spatially  smoothed  covariance  matrix  is 
defined  as  the  average  of  the  subarray  covariances 

1  K 

R=  =  AiRsAi+a2I,  (13) 

k=i 

where  Rs  is  the  modified  covariance  matrix  of  the  signal  given  by 

1  K 

Rs  =  -p  £  DkRsD%.  (14) 

A  k= l 

We  will  show  in  the  following  that  Rs  is  nonsingular.  First,  Rs  can  be  written  as 
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<0 

1 _ 

i 

Rs  =  [I  D2  •  •  •  Dk\ 

1  D 

K- as 

d2 

1  p 

^  s 

Dk 

Let  C  denote  the  Hermitian  square  root  of  j^Rs,  i.e. 


CCH  =  -Rs. 


It  follows  that 

Rs  =  GGh 

where  G  is  a  d  x  Kd  block  matrix  given  by 


(15) 


(16) 


(17) 


G=[C  D2C  •••  DkC). 


(18) 


Clearly,  the  rank  of  Rs  is  equal  to  the  rank  of  G.  Suppose  there  are  q  groups  of  signals  in  d  incoming 
signals,  with  i  =  1,*  •  -  ,q,  correlated  signals  in  each  group.  Rs  must  be  a  block  diagonal  matrix 
with  block  size  k,  i  —  1,  •  •  -  ,q.  We  can  thus  get  a  corresponding  block  diagonal  matrix  C.  Recall 
that  the  rank  of  a  matrix  is  unchanged  after  a  change  of  its  columns.  By  grouping  columns  of 
similar  elements,  we  can  verify  that 


Cipbi  • 

••  Ci^b! 

Qipb/j  • 

•’  ch,lJ°h 

p(G)  =  p 


Cd-lq+l,d-lq+lkd-lq+l  Cd-lq+\,d\>d-lq+l 


(19) 


Cd,d-iq+ibd  ■■■  Cd,dbd 
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where  p  is  a  rank  operator,  c4j  is  the  ijth.  element  of  matrix  C,  and  b;  (i  =  1,  •  •  -  ,d)  is  the  1  x  K 
row  vector  given  by 


b;  =  [1 


Lsm(/32-5,  +  |) 


-3  z-6t  + 1 ) 


3  3  ~ 


,2ndK 


j^sin(P +  j 


(20) 


Each  row  of  matrix  C  has  at  least  one  nonzero  element  because  the  energy  of  each  signal  is  nonzero. 
It  is  observed  that  b8  is  the  transpose  of  the  steering  vector  associated  with  the  center  array.  Since 
the  center  array  is  assumed  to  have  ambiguity  free  array  manifold,  when  K  >  max{/[,  Z2,  •  •  • ,  } , 
all  the  b  vectors  associated  with  all  the  signals  within  a  group  of  coherent  signals  are  thus  linearly 
independent.  Therefore,  G  is  of  full  row  rank  and  the  modified  covariance  matrix  R,s  is  of  full 
rank.  Otherwise,  if  the  center  array  is  not  ambiguity  free,  or  if  I(  <  max{/i,/2,  •  •  • ,lq },  G  cannot 
be  ensured  to  be  of  full  row  rank,  and  neither  can  Rs.  From  Theorem  1  and  the  proof  above,  we 
get  the  following  theorem. 

Theorem  2  SS  can  be  applied  to  an  array  of  arbitrary  geometry  to  obtain  a  full  rank  smoothed 
signal  covariance  matrix  if  and  only  if  an  array  has  an  orientational  invariance  structure,  its  center 
array  has  an  ambiguity  free  structure,  and  the  number  of  subarrays  is  larger  than  or  equal  to  the 
size  of  the  largest  group  of  coherent  signals. 

C  Further  Improvement 

To  get  a  smoothed  nonsingular  covariance  matrix  Rs  by  using  the  SS  technique,  we  need  K  > 
max{l\,l2,  ■  ■  ■  ,lq}.  We  can  further  reduce  the  number  of  subarrays  by  getting  another  K  backward 
subarrays  similar  to  the  case  in  a  linear  array  [23].  Although,  the  Forward- Backward  Spatial 
Smoothing  (FBSS)  [23]  can  always  be  applied  in  a  uniformly  spaced  linear  array.  For  arrays  of 
arbitrary  geometry,  there  is  some  requirements  on  the  geometry  for  successful  implementation  of 
the  backward  method.  We  first  give  the  definition  of  central  symmetry: 

Definition  3  (Central  Symmetry)  The  array  is  central  symmetric  if  it  is  identical  before  and 
after  rotating  180°  about  its  center  of  mass. 

If  an  array  is  central  symmetric,  we  can  get  K  additional  backward  subarrays  by  reversing  the 
order  of  the  subarrays  and  the  order  of  the  sensors  within  each  subarray. 
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Let  r bk(t)  denote  the  complex  conjugate  of  the  output  of  the  kth  backward  subarray  for  k  = 
1,  •  •  • ,  K .  We  have 

r  bk(t)  =  A.D^DLsit))*  +  n  *(t)  (21) 

where  n(f)  is  an  additive  white  Gaussian  noise  vector,  Di  is  a  diagonal  matrix  with  the  ith  diagonal 
element  given  by  e_j27r  a  sin(feP-0.+  2 )  anc[  dKp  }s  the  distance  between  the  first  sensor  in  the  first 
forward  subarray  and  the  first  sensor  in  the  first  backward  subarray.  The  angle  / 3kp  represents  the 
direction  of  the  line  on  which  the  two  sensors  are  located. 

The  covariance  matrix  of  the  fcth  backward  subarray  is  given  by 


R°k  =  AiDkRbsD%  A?  +  a1 1 


(22) 


with 


Rbs  =  EiDis'ityX^Dl)  =  D*LR*sDl 


(23) 


Define  the  spatially  smoothed  backward  subarray  covariance  matrix  Rb  as  the  average  of  these 
subarray  covariance  matrices,  i.e., 


1  K 

R*  =  j'£Rt  =  AlRl.A?  +  e2I, 


(24) 


k= 1 


where 


Rbs=ji:DkRbsD^ 


(25) 


k~l 


and  define  the  forward/backward  smoothed  covariance  matrix  R  as  the  average  of  R  in  (13)  and 
Rb,  i.e., 


R=  =  AiR'Ai+a2!. 


(26) 


It  follows  that 


R, 


Rs  +  Rbs 


(27) 


We  can  show,  in  a  similar  way  as  in  the  case  of  a  linear  array  [23],  that  the  modified  source 
covariance  matrix  Rs  is  nonsingular  as  long  as  2 K  >  max{li,/2?  •  •  -  ,/g}. 
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Ill  Ambiguity  Free  Array  Structure 


To  perform  SS,  we  need  an  ambiguity  free  center  array  manifold.  Also,  to  perform  MUSIC,  we 
further  require  ambiguity  free  subarray  manifolds.  Ambiguity  arises  when  a  steering  vector  can 
be  expressed  as  a  linear  combination  of  other  steering  vectors  in  an  array  manifold  A  [4].  For  a 
uniformly  spaced  linear  array,  rank-1  ambiguity  [4]  cannot  be  avoided  since  the  DOA’s  which  are 
“mirror  images”  with  respect  to  the  array  line,  have  the  same  steering  vector.  This  limits  the  range 
of  DOA’s  estimable  by  a  uniformly  spaced  linear  array  to  within  180°.  Suppose  an  array  has  p 
elements,  then  rank-p  [4]  ambiguities  cannot  be  avoided.  In  this  paper,  an  ambiguity  free  array 
manifold  of  an  array  of  p  sensors  refers  to  rank-(p-l)  ambiguity  free.  Generally,  to  avoid  ambiguity, 
an  array  used  for  high-resolution  DOA  estimation  must  have  a  proper  structure.  An  ambiguity  free 
array  manifold  has  been  assumed  in  several  papers  [5],  [17],  [15].  Our  attempt  is  to  identify  all  the 
situations  in  which  ambiguity  may  arise.  One  of  our  guidelines  in  designing  arrays  is  to  avoid  these 
identified  ambiguities. 

Theorem  3  In  an  azimuth  only  system,  the  necessary  and  sufficient  condition  for  an  ambiguity 
free  three-sensor  array  manifold  is  that  all  these  three  sensors  are  not  on  one  line  and  that  the 
distance  between  any  two  sensors  is  less  than  or  equal  to 

The  proof  is  given  in  the  appendix. 

We  can  see  in  general  that  (a)  rank-1  ambiguity  occurs  not  only  in  uniformly  spaced  linear 
arrays  but  also  in  rectangular  arrays  with  sensors  having  a  uniform  spacing  of  ^  along  either  x-axis 
or  y-axis,  (b)  rank-2  ambiguity  occurs  in  an  array  that  consists  of  two  parallelly  positioned  linear 
arrays  with  an  identical  uniform  sensor  spacing  that  is  larger  than  (c)  rank-3  ambiguity  occurs  in 
an  array  that  consists  of  three  parallelly  positioned  linear  arrays  with  an  identical  uniform  sensors 
spacing  that  is  larger  than  |,  and  (d)  higher  order  ambiguity  occurs  if  more  than  [|]  sensors  are 
on  one  line  in  a  k  sensor  array  or  if  an  array  consists  of  m  parallelly  positioned  linear  arrays  with  an 
identical  uniform  sensor  spacing  that  is  larger  than  |  [yj .  These  situations  are  shown  schematically 
in  Fig.l2(a)-(d).  In  Fig. 12  (b)  and  (c),  the  angles  9  and  a  satisfy  the  following  constraint: 

27 Ty  sin(o)  +  2k-K  =  27T--  sin(0),  k  £  {1, 2,  ■  •  •}.  (28) 

A  A 


ii 


In  Fig. 12  (d),  the  angles  9,  (5  and  a  satisfy  the  following  constraint: 

27r  y  sin(a)  +  2k\-K  =  27T  ^  sin(/f)  +  2&27t  =  2ir  ^  sin(0),  (29) 

AAA 

where  ,&2  €  { 1 , 2,  •  •  •}  and  k\  ^  fc2 . 

To  get  an  ambiguity  free  array  manifold,  it  is  necessary  to  avoid  these  identified  situations.  It  is 
our  conjecture,  although  not  yet  proved  at  this  stage,  that  if  the  spacing  between  two  neighboring 
sensors  of  a  uniformly  spaced  rectangle  array  is  <  this  rectangle  array  is  ambiguity  free.  The 
rationale  is  that  it  is  less  likely  to  have  higher  order  ambiguities  [24]. 


IV  Implementation  and  Experimental  Results 


A  Some  Practical  Considerations 

In  practice,  we  can  perform  FBSS  by  setting  up  a  special  data  matrix.  Specifically,  for  the  nth 
snapshot  we  set  up  the  data  matrix 


u(p,l,n) 

u(p,k,n ) 

w*(l,  k,  n)  •  ■ 

•  «*(1,1,») 

AH(n)  = 

u(p  —  1, 1,  n)  ■ 

•  •  u{p  —  1,  fc,  n) 

u*(2,k,n)  •• 

u*(2,l,n) 

u(l,l,n) 

••  u(l,k,n) 

u*(p,k,n ) 

•  u*(p,l,n) 

where  u(i,j,n)  denotes  the  sample  taken  at  the  ith  sensor  of  the  jth  subarray.  For  the  totality  of 
N  snapshots,  we  can  define  the  overall  data  matrix 


Ah  =  [ AH(l),AH(2),---,AH(N )  . 


(31) 


It  follows  that  the  averaged  smoothed  correlation  matrix  R  (as  defined  in  (26))  is  estimated  as 
follows: 


R  = 


2NK 


:AHA. 


(32) 


As  we  know,  more  robust  results  can  be  obtained  from  data  domain  rather  than  from  covariance 
domain.  We  can  proceed  with  MUSIC  [20]  algorithm  or  MVDR  [7]  [20]  beamforming  algorithm 
based  on  A  instead  of  R. 
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We  also  need  to  choose  an  array  for  applying  SS.  Such  an  array  should  satisfy  all  conditions 
aforementioned.  An  omnidirectional  circular  array  has  been  a  conventional  choice  for  mobile  com¬ 
munications  [1]  [2],  and  there  have  been  active  research  efforts  to  find  a  pre-processing  scheme  for 
the  circular  array  to  handle  the  coherent  interference  [27].  However,  we  can  see  clearly  from  our 
discussion  that  a  circular  array  is  not  an  array  with  orientational  invariance  structure.  Therefore  it 
does  not  satisfy  the  necessary  condition  for  applying  SS.  This  implies  that  the  circular  array  cannot 
overcome  the  coherent  interference  by  using  the  SS  technique. 

B  Experimental  Results 

In  this  section,  we  present  some  simulation  results  on  MUSIC  algorithm  to  show  the  effectiveness 
of  our  SS  and  FBSS.  We  choose  a  square  array,  which  has  an  orientational  invariance  structure, 
central  symmetriy,  and  a  sensor  spacing  less  than 

Example  1:  we  use  a  nine-sensor  linear  array  and  a  nine-sensor  square  array  as  shown  in  Fig. 2, 
both  having  a  spacing  of  0.45A  between  neighboring  sensors.  We  consider  two  narrow-band  coherent 
signals  with  DOA’s  at  70°  and  85°.  The  SNR  is  20  dB.  A  total  of  500  samples  (“snapshots”)  are 
taken  from  the  array.  We  use  SS  as  a  pre-processing  scheme  for  MUSIC.  Fig. 3  shows  that  the 
DOA’s  of  the  two  coherent  signals  are  not  resolved  using  a  linear  array,  whereas  the  square  array 
gives  a  satisfactory  result. 

Example  2:  We  use  the  square  array  shown  in  Fig. 2  to  receive  two  coherent  signals  with 
DOA’s  at  40°  and  50°.  The  SNR  is  20  dB.  A  total  of  500  samples  are  taken.  We  apply  SS  and 
FBSS  separately.  Fig.4  shows  that  the  DOA  estimation  resolution  achieved  by  a  central  symmetric 
array  is  significantly  improved  by  using  the  FBSS  method. 

Example  3:  We  use  a  dense  square  array  of  sixty-four  sensors  as  shown  in  Fig. 7(b).  The  array 
contains  4  subarrays  each  of  49  sensors.  The  spacing  between  two  neighboring  sensors  is  0.45A.  The 
array  receives  four  groups  of  different  signals:  four  coherent  signals  at  20°,  65°,  150°  and  200°,  three 
coherent  signals  at  230°,  250°  and  280°,  two  other  coherent  signals  at  30°  and  300°  and  another  at 
320°.  The  SNR  is  20  dB.  A  total  of  500  samples  are  used.  First,  we  apply  FBSS  and  then  apply 
MUSIC.  Simulation  results  are  shown  in  Fig.5. 

We  found  the  results  obtained  for  a  nonlinear  array  in  an  azimuth-only  system  remain  valid  in 
an  azimuth-elevation  system.  The  following  is  an  example. 
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Example  4:  We  use  the  square  array  shown  in  Fig. 2  to  receive  two  coherent  signals,  one  is  at 
an  azimuth  of  40°  and  an  elevation  of  30°,  and  the  other  is  at  an  azimuth  of  50°  and  an  elevation 
of  60°.  The  SNR  is  20dB.  The  number  of  samples  taken  is  500.  By  using  FBSS  and  MUSIC,  we 
obtain  the  result  in  Fig. 6. 

C  Choices  of  Orientational  Invariance  Structure 

In  this  section,  we  like  to  study  some  guidelines  for  designing  an  optimal  sensor  array  for  SS.  We 
found  that  the  sensor  utilization  rate  is  an  important  factor  for  estimating  DOA’s  of  coherent 
signals  with  SS. 

Definition  4  (Sensor  Utilization  Rate  (SUR)) 

SUR  =  ^nsubarrav  (33) 

^ array 

where  Ylnsubarray  i-s  the  sum  of  the  number  of  sensors  in  each  subarray,  ' and  narray  is  the  total 
number  of  sensors  in  the  whole  array.  Obviously  SUR  >  1,  because  of  possible  overlap  of  subarrays. 

Emample  5:  We  perform  simulations  on  two  64-element  arrays:  (1)  a  dense  square  array  as 
shown  in  Fig. 7(b),  which  has  a  high  SUR  for  a  given  number  of  sensors,  and  (2)  a  hollow  square 
array,  as  shown  in  Fig. 7(a),  which  has  a  low  SUR.  The  dense  square  array  contains  4  subarrays 
each  having  49  sensors.  The  spacing  between  two  neighboring  sensors  is  0.45A.  The  SUR  of  the 
array  is  approximately  3.  The  hollow  square  array  contains  4  subarrays,  each  having  32  sensors. 
The  spacing  between  two  neighboring  sensors  is  0.45A.  The  SUR  is  2.  Both  structures  are  used  to 
estimate  the  DOA’s  of  two  coherent  signals.  The  input  SNR  is  20dB.  We  use  the  FBSS  method. 
The  simulation  results  are  shown  in  Fig. 8.  In  case  (a)  and  (b),  the  two  coherent  signals  are  at  40° 
and  50°.  Both  arrays  can  clearly  identify  the  DOA’s.  In  case  (c)  and  (d),  the  two  coherent  signals 
are  at  45°  and  50°,  only  the  dense  square  array  can  identify  the  DOA’s. 

Our  results  show  that  dense  square  array  structure  is  better  than  the  hollow  square  array 
structure.  Since  both  arrays  have  the  same  number  of  elements,  we  infer  that  the  SUR  is  an 
important  factor  and  needs  to  be  maximized  in  the  array  design. 
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V  Spatial  Smoothing  for  ESPRIT 


Similar  to  MUSIC,  the  ESPRIT  algorithm  [5]  is  an  approach  to  signal  parameter  estimation.  It 
exploits  an  underlying  data  model  at  significant  computational  savings.  The  ESPRIT  algorithm  is 
also  limited  to  estimating  parameters  in  noncoherent  incoming  signals.  The  conventional  SS  can 
be  incorporated  into  ESPRIT  [28],  but  it  requires  the  center  array  to  be  a  uniformly  spaced  linear 
array.  In  this  section,  we  show  that  our  scheme  also  works  for  the  ESPRIT  algorithm  to  estimate 
parameters  in  a  coherent  interference  environment. 

In  the  ESPRIT  algorithm,  we  consider  d  narrow-band  plane  waves  with  incident  angles  0\  -  dj, 

and  wavelength  A,  impinge  on  a  planar  array  of  m  sensors  (m  is  even),  arranged  in  y  doublet  pairs. 
The  displacement  vector  is  the  same  for  each  doublet  pair,  but  the  location  of  each  pair  is  arbitrary. 
The  sensor  output  x(t)  is  given  by 


x(t)  = 


s(t)  +  n(f) 


(34) 


where  n(t)  is  a  white  Gaussian  noise  vector.  A  and  A4>  are  the  steering  matrices  corresponding  to 
the  first  sensors  and  the  second  sensors  in  all  pairs,  respectively.  The  matrix  $  is  a  diagonal  d  x  d 
matrix  of  phase  delays  between  the  doublet  sensors  for  the  d  signals.  The  sensor  output  covariance 
matrix  Rx  is  thus  measured  by 


R 


X 


(35) 


A  full  rank  matrix  Rs  is  assumed  when  the  ESPRIT  algorithm  is  performed.  If  some  of  the 
incoming  signals  are  coherent,  Rs  will  not  be  a  full  rank  matrix  and  the  ESPRIT  will  fail.  The 
spatial  smoothing  technique  we  introduced  in  the  previous  sections  can  then  be  applied  here  to  get 
a  modified  full  rank  signal  covariance  matrix. 

We  consider  each  doublet  sensor  pair  in  the  array  used  by  ESPRIT  algorithm  as  one  element. 
Then  the  array  consists  y  elements.  If  this  array  has  an  orientational  invariance  structure  with  K 
subarrays  and  the  corresponding  center  array  has  an  ambiguity  free  structure,  the  sensor  output 
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at  the  kth  subarray  is  given  by 


xjfe(i)  = 


/ 

\ 


Ai 


\ 

Dks(t)  +  n(t). 

/ 


(36) 


Matrix  Dk  is  a  diagonal  dxd  matrix  of  the  phase  delays  in  the  form  given  in  (11).  The  corresponding 
covariance  matrix  RXk  is  given  by 


R 


ock 


( 


Ai 


V 


\  ( 

DkRsDk 

/  V 


Ai 


V 

+  a2I. 


A.  smoothed  output  covariance  matrix  Rx  can  thus  be  defined  as 


(37) 


1  K 

=  R xk  ~ 


i— 1 


Ai  ) 

RS 

M 

,  Ai*  j 

j 

H 


+  vl 


(38) 


where  Rs  is  the  modified  signal  covariance  matrix  as  defined  in  (14).  As  proved  in  Section  II,  Rs 
is  of  full  rank  if  K  is  larger  than  or  equal  to  the  size  of  the  largest  group  of  coherent  signals.  We 
can  now  successfully  perform  ESPRIT  based  on  Rx.  We  can  also  use  FBSS  to  further  reduce  the 
number  of  sensors  required  and  to  improve  the  estimation  resolution  if  the  array  of  y  element  is 
central  symmetric. 

Although  SS  enables  ESPRIT  to  estimate  DOA’s  in  a  coherent  interference  environment,  the 
estimation  is  still  limited  to  identifying  DOA’s  within  180°  in  an  azimuth  only  system.  Hence,  in 
terms  of  performance  robustness  to  DOA’s,  our  SS  is  more  effective  for  MUSIC  than  for  ESPRIT. 

Example  6:  A  twelve  sensor  array  shown  in  Fig. 9  is  used  in  this  example  to  receive  two 
coherent  signals  at  70°  and  80°.  This  array  consists  of  two  overlapping  nine-sensor  square  arrays. 
Each  sensor  in  one  square  array  and  its  counterpart  in  another  form  a  doublet  pair.  These  nine 
doublet  pairs  form  an  array  which  has  orientational  invariance  structure  and  is  central  symmetric. 
The  spacing  between  two  neighboring  sensors  is  0.45A.  The  doublet  spacing  for  ESPRIR  is  0.45A. 
The  SNR  is  20dB.  A  total  of  2000  trials  are  run.  A  histogram  of  the  results  is  given  in  Fig. 10.  We 
apply  FBSS  first  and  then  applied  the  ESPRIT.  The  two  angles  are  clearly  identified. 
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VI  Conclusions 


To  significantly  improve  performance  robustness  in  DOA  estimation  and  in  adaptive  beamforming, 
we  developed  techniques  for  applying  SS  on  arrays  of  arbitrary  geometry,  thus  making  MUSIC, 
ESPRIT  and  adaptive  beamformers  operative  in  a  coherent  interference  environment.  In  order 
to  apply  SS  to  an  array  of  arbitrary  geometry,  this  array  must  have  an  orientational  invariance 
structure  and  its  center  array  must  be  ambiguity  free.  Also  the  number  of  subarrays  must  be  greater 
than  or  equal  to  the  largest  number  of  mutually  coherent  signals.  To  apply  SS  in  conjunction 
with  MUSIC,  all  the  subarrays  must  also  be  ambiguity  free,  and  the  number  of  sensors  in  each 
subarrays  must  be  larger  than  the  number  of  incoming  signals.  For  ESPRIT,  two  identical  arrays  (or 
subarrays)  separated  by  a  displacement  vector  are  used  each  satisfying  the  conditions  for  applying 
SS  and  MUSIC. 

When  a  nonlinear  array  is  central  symmetric,  the  FBSS  can  be  used  and  it  outperforms  the 
regular  SS  in  terms  of  improved  efficiency  and  estimation  resolution. 

We  proved  the  necessary  and  sufficient  conditions  for  a  three-sensor  array  manifold  to  be  am¬ 
biguity  free.  We  identified  several  situations,  for  higher  order  sensor  array  manifolds,  in  which 
ambiguity  may  arise.  It  is  necessary  to  avoid  the  identified  ambiguities  in  designing  ambiguity  free 
center  arrays  and  subarrays. 

In  practice,  we  found  that  we  can  choose  a  square  array  with  a  sensor  spacing  less  than  | 
to  meet  all  the  conditions  required  for  applying  SS.  Simulation  results  also  show  that  for  DOA 
estimation  of  coherent  signals  using  SS,  a  square  array  has  a  prefered  geometry  in  terms  of  the 
DOA  estimation  resolution  and  performance  robustness. 

VII  Appendix 

Proof  of  Theorem  1: 

If  part: 

If  sensors  A,  B  and  C  are  not  on  one  line  and  their  mutual  distance  is  less  than  |,  without  loss 
of  generality,  we  let  sensor  A  be  the  first  sensor  in  the  array,  B  the  second  and  C  the  third.  The 
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steering  matrix  of  the  array  has  the  form 


V  = 


1  1  1 

ej<t>  i(#i)  ej<t>  1(^2)  ej<t>i(9z) 

ei4>2(6i)  ej<t>  2(02)  ei<t>2{h) 


(39) 


where  <f>  denotes  phase  delay.  If  the  distance  between  any  two  sensors  is  <  j,  the  phase  delay  fa(9i) 
and  fa(9i),i  =  1,2,3,  are  real  numbers  from  (— 7r,7r). 

Note  that  the  steering  matrix  of  the  array  corresponding  to  three  incoming  signals  at  different 
angles  is  a  special  case  of  the  general  array  in  Lemma  2  of  [24].  By  Lemma  2  in  [24],  V  is  nonsingular 
with  possible  exception  in  one  of  the  following  three  situations: 

(1)  When  fa(9i)=fa(92),  be.  the  two  incoming  signals  are  symmetric  with  respect  to  the  line 
on  which  sensors  A  and  B  are  located.  Note  that 


deHy) 


0  1  0 

ei<t>\(9i)  _  gj^i(02)  2)  ei4>\ (#3)  _  ei4>i (#2) 

ei<h(6i)  —  ei<t>2(h)  gjfaifa)  ej<h(h)  _  ej4> 2(^2) 
ej<f>i(8i)  _  ei0i (^2) 

ej<h(6i)  _  gj $2 (^2 )  ei4> 2(63)  _  ej<t> 2(^2) 


(40) 


(41) 


When  <j>  1(61)  —  fa(92),  det(V)  =  0  if  and  only  if  fa{9\)  =  fa(92)  or  </>i(03)  =  <f)  i{92).  Since  these 
sensors  are  not  on  one  line,  if  fa{9\)  =  (j>  \(92),  we  have  (^2(^1)  7I  </> 2(92).  Since  #i,<?2  and  #3 
are  three  different  angles,  when  #1  and  62  are  symmetric  with  respect  to  the  line,  63  and  02  can 
not  be  symmetric  to  the  line,  i.e.  if  <^1(^1 )  =  4>  1(^2),  then  we  get  <^1(^3)  7^  fa(92).  Thus,  when 
fa{9\)  =  4>i(92),  the  matrix  V  is  nonsingular. 

(2)  Similarly,  we  can  prove  that  when  fa{9\)  =  fa{92),  the  matrix  V  is  nonsingular. 

(3)  When  fa{9{)  -  fa (02)  =  fa(9i)  -  fa(02),  i.e.  cf> i(6>i)  -  fa(9i)  =  fa(92)  -  fa(92),  9X  and  02 
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are  symmetric  with  respect  to  the  line  connecting  sensors  B  and  C .  Note  that 


det{V) 


e~i4>2{9 1)  (^2)  g  j(l32  ($3 ) 

(^1 )— i<^>2  (^1 )  ejM<h)-j<t>2(h)  ei4>i(9i,)-i4>2{h) 

1  1  1 

^e-j<t>2(0l)  _  g~j4>2  ($2  )  ^gi^l  (^2  )  j4>2  ($2)  _  gj01  (^3  )— J02  (^3  )  ^ 


(42) 

(43) 


When  </>i(#i)  —  0 2(^1)  =  ^1(^2)  —  $2(62),  det(F)  =  0  if  and  only  if  e~ f^2(01)  =  e— i02 (^2)  or 
ei<l>i(62)- 3^2(62)  —  ej<^>i (<93)— j<^2 (^3) _  Since  the  mutual  distance  between  A,  B  and  C  are  less  than 
2  >  02(0i),  ^2(^2))  0i(#2)  —  ^2(^2)  and  <j)\(6z)  —  ^2(^3)  are  all  real  numbers  in  (— 7r,7r).  e~^2^  — 
e-iM*i)  if  and  only  if  02(0i)  =  02(02)-  jf  and  only  if  <^(02)  - 

^2(^2)  =  ^1(^3)  -  4> 2(^3)- 

Since  A,  B  and  C  are  not  on  one  line,  if  6\  and  62  are  symmetric  to  the  line  connecting  B  and 
C,  they  can  not  be  symmetric  to  the  line  connecting  A  and  B  or  A  and  C.  i.e.,  if  <t>i{0\)  -  02(#i)  = 
0i(02)  -  0 2(^2),  we  have  ^2(^1)  7^  02 ($2)-  Since  ,  02  and  63  are  three  different  incoming  angles, 
if  9\  and  62  are  symmetric  to  the  line  connecting  B  and  C,  62  and  83  can  not  be  symmetric  to  the 
line,  i.e.,  if  0i(0i)  -  02(0i)  =  0i(02)  -  02(^),  we  have  0i(02)  -  #  M^)  ~  02(03).  Thus, 

when  0i(0i)  -  02(#i)  =  0i(02)  -  02(^2),  the  matrix  V  is  nonsingular. 

Therefore,  we  conclude  that  all  the  three  situations  which  cause  the  singularity  of  the  matrix 
in  Lemma  2  of  [24]  will  not  cause  the  singularity  of  three-sensor  steering  matrix  if  three  sensors 
are  not  on  one  line  and  their  mutual  distance  is  less  than  Therefore  the  matrix  V  is  full  rank. 

If  the  spacing  between  any  two  of  the  three  sensors  is  not  larger  than  j,  and  there  is  at  least 
one  pair  in  these  three  sensors  with  a  spacing  of  then  the  only  situation  that  the  phase  delay 
< f>i(0i )  and  02(0*),*  =  1,2,3,  are  not  all  in  (— 7r,7r)  is  when  one  of  the  incoming  signals  is  from 
the  direction  parallel  to  a  line  on  which  the  two  sensors  with  spacing  j  are  located.  The  other 
two  signals  can  be  either  from  the  opposite  direction  or  from  other  directions.  If  one  of  the  other 
two  signals  is  from  the  opposite  direction,  it  can  be  easily  proved  that  the  corresponding  steering 
matrix  is  full  rank.  If  the  other  two  signals  are  from  the  two  other  different  directions,  then  one  of 
0n(0*),  n—  1*2,  *  =  1,2,3  is  equal  to  7 r  and  the  rest  are  real  numbers  from  (— 7r,7r).  Similarly,  we 
can  prove  that  the  matrix  V  is  of  full  rank. 

Only  if  part: 
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If  the  conditions  in  Theorem  1  are  not  satisfied,  rank-1  or  rank-2  ambiguity  occurs  for  some 
incoming  signals.  These  situations  are  shown  schematically  in  Fig.ll(a)(b).  In  Fig.ll(a),  the 
relation  between  9  and  a  is 

2TT-sin(0  -  a)  +  k2n  —  2ir^-sin(0  +  a)  k  €  {1, 2,  ■  •  •}.  (44) 

A  A 

In  Fig. 11(b),  the  relation  between  9  and  a  is 

2ic~sin(a)  +  k2ir  =  2ir^sin(^- -  0),  ke{  1,2,  •••}.  (45) 

A  A  2 


Proof  of  Lemma  1: 

If  part: 

The  proof  is  obvious  and  is  omited. 

Only  If  part: 

If  B  =  AC,  A  =  [a{9\),  •  •  • ,  a(9d)]  and  B  —  [&(#i),  •  •  • ,  b{9d)]  and  also  assume  C  is  not  a  diagonal 
matrix,  i.e.  it  has  non-zero  element  c/m  for  l  m ,  then  the  steering  vector  b(9m )  is 

d  d 

b(9m)  —  ZCim<0i)  =  Clm&iPl)  T  )  ]  Cjm^(^j)'  (46) 

i=l  i—1  ,i^l 

This  means  that  b(0m )  is  a  function  of  variable  9\,  which  contradicts  to  the  definition  that  b(9m)  is 
only  a  function  of  9m.  Thus  the  assumption  that  C  is  a  non-diagonal  matrix  is  false.  C  has  to  be 
a  diagonal  matrix  with  cu  —  7$f\- 

Proof  of  Lemma  2: 

If  part: 

Obviously,  B  can  be  any  of  {Ai,  A2,  ■  ■  ■ ,  A^-}. 

Only  if  part: 

If  each  A{  can  be  mapped  to  a  steering  matrix  B,  by  definition  there  exist  Ci,  Cj  such  that 
Ai  =  BCi,  Aj  —  BCj.  By  Lemma  1,  Ci  is  a  diagonal  matrix.  So  1  exists  and  is  also  a  diagonal 
matrix.  We  have  Aj  =  AiC^Cj.  Let  Ctj  =  C~lCj,  Cij  is  the  product  of  two  diagonal  matrices. 
So  Cij  is  also  an  diagonal  matrix.  Aj  =  AtC,y. 
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Figure  1:  Orientational  invariance  sensor  array  geometry 


d 


Figure  2:  A  nine-sensor  square  array  with  spacing  d 
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Figure  3:  SS  and  MUSIC  for  DOA  estimation  of  two  coherent  signals  at  70°  and  85° 


Figure  4:  DOA  estimation  of  two  coherent  signals  at  40°  and  50°  by  using  a  nine  sensor  square 
array 
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DOA(degree) 

Figure  5:  DOA  estimation  of  four  groups  of  coherent  signals  at  (20°, 65°, 150°, 200°),  (230, 250°, 280°), 
(30°, 300°)  and  320°  based  on  a  sixty  four-sensor  square  array 


el6''allon  °  °  azimuth 


Figure  6:  DOA  estimation  of  two  coherent  signals  at  an  azimuth  of  40°  and  an  elevation  of  30°, 
and  at  an  azimuth  of  50°  and  an  elevation  of  60°,  respectively 
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(a) 


Figure  7:  (a)  A  sixty  four-sensor  dense  square  array  with  four  overlapping  dense  square  subarrays 
of  forty  nine-sensors  (b)A  sixty  four-sensor  hollow  square  array  with  four  overlapping  hollow  square 
subarrays  of  thirty  two-sensors 


Figure  8:  A  sixty  four-sensor  hollow  square  array  is  used  in  (a)  and  (c),  A  sixty  four-sensor  dense 
square  array  is  used  in  (b)  and  (d) 
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(a)  rank-1  ambiguity 


(b)  rank-2  ambiguity 

Figure  11:  Three-sensor  array  structures  that  can  cause  ambiguities 
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(a)  rank-1  ambiguity 


(b)  rank-2  ambiguity 


(c)  rank-3  ambiguity 
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(d)  high  order  ambiguity 

Figure  12:  high  order  array  structures  that  can  cause  ambiguities 
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